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PREFACE 


This  paper  was  presented  at  the  2lBt 
Annual  National  Forum  of  The  American 
Kelicopter  Society  on  13  May  I965.  It 
describes  some  approaches  to  modeling 
for  systems  analysis  or  cost-effectiveness 
studies. 
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IMIRODUCTIOH 


Elements  of  Cost-Sffectiveness  Analysis 

Cost-effectiveness  studies  are  an  aid  to  resource  allocation 
decisions.  The  principal  elements  of  a  cost-effectiveness  analj^sis  are; 

Objective(s)  (functions  to  be  accomplished) 

Alternatives  (feasible  ways  of  achieving  the  desired 

military  capability  or  accomplishing  the  function) 

Costs  of  resource  requirements  for  each  alternative 

Models  (sets  of  mathematical  or  logical  relationships 
among  the  objectives,  alternatives,  environment 
and  resources) 

Criterion  for  choosing  the  preferred  alternative 
The  criterion  specifies  the  relation  between  the  measure  of  effective¬ 
ness  and  the  measure  of  cost  that  will  result  in  preference.  The  effective¬ 
ness  measure  should  express  the  extent  to  which  the  objective  is  being 
accomplished.  The  measure  of  cost  must  be  consistent  with  the  overall 
framework  of  the  resource  allocation  problem,  e.g,,  net  future  total 
peacetime  cost  of  ownership  for  N  years. 

An  extensive  discussion  of  the  criterion  of  preference  is  beyond 
the  scope  of  this  paper;  Chapter  9  of  Hitch  and  McKean,  The  Economics  of 
Defense  In  the  Nuclear  Age,  ^  covers  the  criterion  problem  admirably. 
However,  some  problems  in  selection  of  criteria,  and  in  selection  of  a 
preferred  alternative  within  the  constraints  of  a  given  criterion,  are 
examined  in  the  final  section  of  the  paper,  "Cost-Effectiveness  Model," 
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Regardless  of  the  specific  criterion  to  be  applied,  the  effectiveness 
and  cost  of  each  of  the  alternatives  must  be  estiraated  and  related.  These 
estimates  are  made  by  means  of  several  types  of  logically  interrelated 
models.  The  emphasis  of  this  paper  is  on  the  conceptual  design  and  function 
of  the  principal  types  of  models  used  in  cost-effectiveness  analysis. 

Role  of  Models 

All  models  used  in  cost-effectiveness  analysis  are  formalized  relations 
among  abstractions  from  the  real  world.  They  perform  a  dual  purpose  of 
(a)  reducing  the  problem  to  manageable  proportions,  and  (b)  identifying 
those  variables  and  parameters  that  are  significant  to  the  decision 
process . 

The  problem  in  any  cost-effectiveness  analysis  is  how  to  get  from 
"here’ to- there";  from  the  estimate  of  effectiveness  to  the  estimate  of 
cost  of  each  alternative^  and  from  the  effectiveness  and  costs  of  each 
alternative  to  selection  of  the  preferred  alternative  for  accomplishing 
the  objective.  This  progression  cannot  be  accomplished  directly,  but 
is  made  possible  by  four  principal  types  of  interrelated  models; 
Effectiveness  models 
System  and  organization  models 
Cost  models 

Cost-effectiveness  models 

The  interrelation  of  these  models  is  illustrated  in  Figure  1. 

An  effectiveness  model  relates  the  measures  of  effectiveness  on 
the  functions  to  be  accomplished  (the  objective)  to  measures  of  performance 


Fig.  1 — Principal  Categories  of  Models  in  Cost-Effectiveness  Analysis 
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in  the  context  of  an  operational  concept. 


A  system  and  organization  model  relates  the  measure  of  performance 
to  the  physical  specifications,  quantities,  and  activity  rates  of  a  system 
and/or  organization  (equipment,  personnel,  and  procedures)  consistent  with 
the  operational  concept. 

A  cost  model  relates  the  quantities,  specifications,  and  activity 
rates  of  the  system,  and  of  the  organization  that  operates  and/or  supports 
it,  to  measures  of  cost  through  cost  factors  and  estimating  relationships. 

A  criterion  or  cost-effectiveness  model  relates  cost  and  effectiveness 
of  each  alternative  to  aid  in  the  selection  of  the  preferred  alternative. 

The  concepts  of  design  and  the  function  of  the  models  as  used  in  the 
analytical  process  are  illustrated  in  the  following  sections  (for  a  com¬ 
prehensive  discussion  of  modeling,  see  "Guide  for  Reviewers  of  Studies 
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Containing  Cost-Effectiveness  Analysis.") 

DESIGN  AHD  application  OF  MODELS 

Within  this  section,  a  set  of  models  is  described  that  were  designed 
for  the  estimation  of  effectiveness  and  cost  of  alternative  aircraft 
systems  for  the  same  objective.  The  models  were  developed  as  part  of 
RAC's  research  in  cost-effectiveness  methodology. 

The  Objective 

A  hypothetical  problem  is  used  to  illustrate  the  application  of  the 
models.  In  the  exanple,  a  military  requirement  to  accomplish  a  set  of 
tactical  missions  had  been  identified,  for  which  one  of  several  different 
aircraft  was  to  selected  as  the  principal  item  of  materiel.  The 
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criterion  is  the  lowest  peacetime  cost  for  a  wartime  capability  to 
accomplish  the  mission  requirement  on  a  sustained  basis. 

An  analysis  of  the  objective  (operational  requirement)  revealed  the 
following  information: 

There  were  several  different  mission  profiles.  All  missions  required 
return  to  the  base  of  operations  after  completion.  A  requirement  for  a 
specific  quantity  of  missions  per  day  existed  in  a  number  of  military 
units  located  in  different  operating  environments.  All  of  the  missions 
had  to  be  flown  on  extremely  short  notice.  Demands  for  sorties  were 
random  in  time,  and  there  were  no  priority  differences  among  missions  so 
requested.  The  different  profiles  occurred  with  different  relative 
frequency  among  military  units. 

For  each  of  the  aircraft,  performance,  reliability,  maintainability, 
and  cost  estimates  were  available,  and  it  had  been  determined  by  other 
means  that  each  aircraft  could  accomplish  the  mission  tasks. 

Support-personnel  costs  and  factors,  peacetime  activity  rates,  and 
other  materiel  planning  factors  were  available. 

Conceptual  Design  of  the  Models 

The  design  of  the  models  was  based  on  the  following  overall  concept: 

For  the  measurement  of  effectiveness,  to  be  expressed  as  the  number 
of  missions  per  day  completed,  the  wartime  operational  requirements  were 
to  be  translated  into  demands  for  flying  time  as  a  function  of  aircraft 
performance  (effectiveness  model).  The  demands  for  flying  time' were  to 
be  translated  into  numbers  of  aircraft  and  personnel  as  a  function  of  the 
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aircraft  support  requirements  and  policies  (systems  model).  The  resource 
requirements  for  aircraft  and  their  support  to  provide  a  vartime  capability 
were  to  be  translated  into  peacetime  costs  (cost  model).  The  effectiveness 
and  systems  models,  generating  physical-resource-requireraent  data,  were  to 
simulate  the  operations  to  adequately  represent  at  least  some  of  the  more 
important  interdependencies.  A  flow  diagram  illustrates  the  conceptual 
design  of  the  three  models  in  Figure  2. 

The  models  were  to  be  so  designed  that  all  repetitive  operations 
would  be  performed  by  a  digital  computer  and  all  one-time  operations  by 
hand.  This  was  an  arbitrary  allocation  .i stated  by  time  and  manpower 
constraints.  Sequential  operations  had  to  oe  minimized  to  limit  potential 
bottleneclis  in  the  process  of  analysis.  Computer  schedule  constraints 
limited  automatic  Iteration. 


Description  of  Models 

Iffectiveness  Model.  The  function  of  this  model  was  to  rebate 
effectiveness  to  performance  under  simulated  operational  uses.  Its  out¬ 
put —  ordered  demands  for  flying  t,ime--was  derived  irora  two  principal  con- 
sideratioriB;  the  time  required  to  fly  each  mission  profile,  and  the  time 
distribution  of  requests  for  missions. 

The  relative  frequency  of  mission  tyjies  at  eech  military  unit  was 
calculated  manually  and  its  operating  environment  noted.  The  time  (and 
fuel)  to  fly  each  mission  was  calculated  manually  from  aircraft  perfor¬ 
mance  data  or  each  planned  flight  profile,  for  each  aircraft,  at  each 
environcift;:.  il  conuinion  (altitude  and  temperature).  The  flight  times 


Fig.  2-Flow  Diagrom  of  Effectiveness, 


(and  fuel)  for  each  mission/environmental  condition/aircraft  combination 
were  then  matched  with  the  military  unit/relative  taission  frequency/environ- 
menta ;  condition  combinations  to  form  cumulative  frequency  distributions 
for  each  aircraft/military  unit  combination. 

To  simulate  the  operations,  random  numbers  were  selected  to  draw 
requests  for  missions,  i.e.,  flight  times,  which  were  ordered  in  a  queue 
(see  Figure  3). 

A  "dispatcher"  in  the  computer  program  assigned  aircraft  from  the 
system  model's  pool  to  flight  requests  under  a  choice  of  several  policies, 
e.g.,  various  options  on  the  length  of  the  operations  day.  At  the  end 
of  each  simulated  day,  unfulfilled  mission  requests  were  canceled. 

A  "bookReeping"  computer  subroutine  recorded  totals  and  calculated 
averages  of  numbers  of  missions  requested,  missions  completed,  delays, 
flight  time,  and  fuel  consumption  for  each  model  run  (simulated  months 
of  operations). 

Systems  Model.  This  model  was  connected  to  the  effectiveness  model 
via  the  "dispatcher".  The  systems  model  related  the  flying-tiiiie  demands 
and  the  resulting  maintenance  requirements  to  numbers  of  aircraft  and 
maintenance  personnel  needed  to  accomplish  the  missions  as  a  function  of 
the  reliability  and  maintainability  characteristics  of  the  aircraft. 
Limitations  of  crew  and  spare-parts-availability  were  excluded  from 
this  model  because  they  were  exogenous  constraints,  i.e.,  not  directly 
dependent  on  the  postulated  aircraft  designs. 

Estimates  of  reliability  and  maintainability  for  each  aircraft  at 


Fig.  3— Effectiveness  Model:  Aircraft  Operotionol  Requirements 


each  operational  condition  were  analyzed  in  detail  to  compute  manually 
the  intervals  of  scheduled  maintenance  man-hours  (accrued  on  a  flying-hour 
basis  but  performed  after  a  sortie)  and  distributions  of  unscheduled 
maintenance  man-hours  per  sortie.  These  times  ranged  from  zero  (no  repair) 
to  infinity  (lost  in  fliglit). 

For  sensitivity  analysis^  several  distributions  of  unscheduled  main¬ 
tenance  were  created  to  reflect  assumptions  about  the  impact  of  higher- 
echelon  repair  work  as  more  effort  or  delay. 

The  unschedu3.ed  maintenance  subroutine  could  also  simulate  wartime 
losses  of  aircraft  by  adjustment  of  the  "lost- in- flight"  percentage  to 
reflect  assumptions  about  effects  of  vulnerability  to  enemy  action  and 
wartime  accidents. 

The  flow  diagram  of  the  systems  model  is  shown  in  Figure  4.  To 
simplify  the  model,  ground  aborts  were  excluded — it  was  assumed  that  if 
an  aircraft  were  available,  it  would  take  off. 

Several  options  for  maintenance  policies  were  designed  into  the 
model  to  facilitate  sensitivity  analysis.  For  the  simulation  of  continuous 
operations,  both  scheduled  and  unscheduled  ma-intenance  were  simulated.  To 
test  for  sensitivity  to  unscheduled  raainoenance,  the  effectiveness  model 
assigned  aircraft  only  during  a  selected  part  of  the  day,  scheduled 
maintenance  was  assumed  to  take  place  during  the  rest  of  the  day,  and 
only  the  unscheduled  maintenance  routine  was  used.  Maintenance  personnel 
could  be  assigned  under  several  policies  reflecting  their  availability 
and  workload.  Aircraft  turnaround  times  were  another  option. 
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Pig.  4 — Systems  Model;  Aircraft  end  Maintenance  Requirements 
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ASSIGNMENT  POLICY 


After  each  aircraft's  assignment,  the  records  of  its  simulated 
cumulative  engine  time  were  interrogated  to  determine  if  scheduled 
maintenance  were  due  after  the  sortie.  A  random  number  was  used  to 
select  the  amount  of  unscheduled  maintenance,  due  after  the  sortie, 
from  the  cu’iUlative  distribution.  Unscheduled  and  scheduled  maintenance 
were  additive. 

If  no  maintenance  were  called  for,  the  aircraft  was  returned  to 
availability  status  via  an  optional  turnaround  time.  If  maintenance 
were  due,  the  aircraft  entered  a  queue  and  awaited  assignment  of 
maintenance  personnel  according  to  the  policy  selected,  e.g.,  to  assign 
one  available  mechanic  to  any  job  less  than  two  man-hours,  two  available 
mechanics  to  jobs  for  two  to  six  man-hours,  etc.  The  computer  model 
calculated  the  downtime  according  to  the  maintenance  man-hours  required 
and  the  maintenance  men  assigned.  Following  maintenance,  aircraft  were 
returned  to  availability  status.  If  maintenance  were  infinite,  no 
mechanics  were  assigned  and  the  aircraft  was  entered  in  the  tally  as 
lost  (see  Figure  5), 

The  computer  programs  of  the  effectiveness  and  systems  models  are 

described  in  a  PAG  paper  now  in  preparation:  "A  Simulation  Model  for 
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Vehicle  Operation  with  a  Set  of  Stochastic  Conditions." 

Cost  Model.  The  systems  model  furnished  estimates  on  the  resources 
(aircraft,  personnel,  fuel,  hc’irs,  etc.)  required  to  provide  the  simu¬ 
lated  wartime  capability  expressed  in  the  measure  of  effectiveness. 

The  peacetime  costs  of  this  capability  were 


computed  with  the  aid  of 


Aircraft  Maintenance  and  Personnel  Requirements 


cost  models  that  derived  total  system  costs  from  the  simulated  resource 
requirements  by  factors  and  estimating  relations. 

To  conserve  time,  costing  proceeded  in  parallel  with  the  simulations 
in  the  effectiveness  and  systems  models.  Research  and  development  costs 
were  estimated  separately  by  hand.  Cost  sub-models  were  used  to  generate 
separate  schedules  of  costs  for  aircraft  systems  and  for  maintenance  per¬ 
sonnel  so  that  costs  of  any  combination  of  these  two  principal  resources 
could  be  determined  without  delay. 

Aircraft  system  acquisition  and  operational  cost  factors  differea 
for  each  aircraft  type.  They  included  the  complete  aircraft  itself, 
ground- support  equipment,  parts,  and  fuel  consumption.  Planning  factors 
for  maintena.iCe  and  replacement  of  ground- support  equipment,  initial 
stocks  of  spares,  peacetime  attrition  (expressed  as  aircraft  replaced 
per  100,000  flying  hours),  maintenance  float,  and  training  aircraft 
were  common  to  all  designs. 

The  cost  factors  were  entered  into  the  computer  sub-models,  which 
calculated  the  initial  investment  and  annual  operating  costs  for  the 
given  numbers  of  aircraft,  years  of  operation,  annual  flying  hours,  and 
attrition  factors,  and  then  added  to  these  the  costs  of  research  and 
development.  The  sub-models  for  initial  investment  and  annual  operating 
costs  are  illustrated  in  Figures  6  and  7,  respectively. 

Preparation  of  the  input  data  was  a  manual  process  often  requiring 
considerable  analysis  and  use  of  estimating  relations  (the  types  of 
cost  model  input  data  are  listed  in  Figure  8) . 
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Fig.  6 — Cost  Model  Subroutine  for  Calculating  Initial  Investment  Costs 


Initial  Investment  Cost  Sub-Models 
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Fig.  8 — Cost  Model  Input  Doto 
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TYPES  OF  INPUTS  TO  COST  ANALYSIS  MODEL 


Fig.  7“Cosl  Model  Subroutine  for  Calculating  Annual  Operating  Costs 
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Annual  Operating  Cost  Sub-Models 


For  the  estimation  of  the  spares-cost,  per  flying  hours,  the  scheduled 
and  unscheduled  maintenance  data  prepared  for  i.he  system  model  were  used. 

A  detailed  description  of  the  cost  model  computer  program  will  he 
published  in  a  futture  RAC  paper,  now  in  preparation;  "An  Indivldiial  Syatera/ 
Organization  Cost  Model.”  — ' 

Sensitivity  A.nalysis.  In  each  of  the  models  discussed  above,  calcu¬ 
lations  were  repeated  with  small  changes  in  input  values,  and  with  different 
simulation  rules  (option.^  or  policies)  to  determine  the  sensitivity  of  the 
results  to  such  changes  (see  Figure  9)*  In  this  manner,  assumptions  were 
identified  which  must  be  explicitly  included  in  the  inforraa-cion  presented 
for  the  selection  of  the  preferred  alternative. 

Cost-Effectiveness  Model.  To  select  the  preferred  alternative,  a 
criterion  is  applied  relating  cost  and  effectiveness  in  some  specified 
manner.  As  mentioned  earlier,  the  preferred  alternative  was  to  be 
selected  on  the  basis  of  the  lowest  peacetime  cost  for  a  specified  level 
of  effectiveness,  i.e.,  wartime  capability. 

The  cost-ei'fectiveness  model  is  not  a  computer  program;  rather,  it 
is  a  type  of  decision  model.  Without  going  into  a  lengthy  discussion 
of  the  theoretical  aspects  of  resource  allocation  decisions,  a  series  of 
simplified  illustrations  -will  be  used  to  describe  the  cost-effectiveness 
model  -  the  raison  d'etre  of  the  computer  models  discussed  above. 

Simulations  by  the  effectiveness  and  system  models  showed  that  the 
same  level  of  effectiveness,  i.e.,  miGsions  completed  per  day,  could  be 
obtained  for  each  of  the  alternatives  with  reverai  different  comb'' nations 


Fig.  9'— SensitiYitjr  Parumeters 
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of  maintenance  personnel  and  aircraft  systemG.  T^iis  limited  substitutability 
is  illustrated,  albeit  exaggerated,  in  Figure  10  for  Alternative  2.  Each 
point  on  the  curve  represents  a  combination  of  aircraft  systems  and  maixiten- 
ance  personnel  which  will  yield  the  same  effectiveness.  Figure  LI  shows 
the  equal-effectiveness  curves  of  all  three  alternatives:  each  point  on 
each  of  the  three  curves  represents  the  same  cf  missions  completed 

per  day. 

The  cost  models  furnished  schedules  of  cost  for  aircraft  systems  and 
for  maintenance  personnel,  respectively.  Different  combinations  of  mainten¬ 
ance  personnel  and  aircraft  systems  can  be  obtained  for  a  given  budget. 

Figure  12  shows  these  combinations  for  several  budget  levels  for  one  of 
the  alternatives.  Each  point  on  a  given  line  represents  a  combination  of 
maintenance  men  and  aircraft  systems  available  for  the  budget  expressed 
by  that  line.  The  slope  of  the  line  is  determined  by  the  relative  costs 
of  these  two  factors. 

Figure  13  illustrates  the  combinations  of  maintenance  personnel 
and  aircraft  systems  for  each  of  the  alternatives  at  two  budget  levels. 
Because  of  differences  in  costs  among  the  three  alternative  aircraft 
systems,  the  relative  costs  of  maintenance  personnel  and  aircraft  systems, 
i.e.,  the  slopes  of  the  budget  lines,  differ  for  each  alternative. 

iTi-  efch  alternative,  many  different  combinations  of  factors  can  be 
obtained  for  the  same  cost,  and  many  different  combinations  will  yield 
the  same  level  of  effectiveness,  but  only  one  combination  is  the  most 
efficient  for  each  alternative.  This  concept  is  illustrated  in  Figure  It, 


Fig.  lO—Equal -Effectiveness  Curve  of  One  Alternative 
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which  shows  the  equal-effectiveness  curve  of  Alternative  2  (as  shown  in 
Figure  10)  and  the  equal-cost  line  for  Alternative  2  which  is  tangent 
to  that  curve.  The  combination  of  maintenance  personnel  and  aircraft 
systems  defined  by  this  point  of  tangency  of  the  equal-effectiveness  curve 
and  the  equal-cost  line  is  the  most  efficient  for  Alternative  2.  Any 
other  combination  would  cost  more  or  accomplish  less. 

Figure  15  is  the  cost-effectiveness  model.  It  shows  all  three 
alternatives  compared  at  the  same  level  of  effectiveness.  Alternative  1 
will  accomplish  the  objective  at  the  highest  cost;  Alternatives  2  and  3# 
respectively,  at  less  cost  than  Alternative  1.  Alternatives  2  and  3  have 
the  same  cost  and  effectiveness  and  differ  only  in  the  ratio  of  maintenance 
personnel  to  aircraft  systems.  Given  the  original  cri-^erion,  no  preference 
between  Alternatives  2  and  3  can  be  stated;  other  criteria,  e.g.,  implicit 
different  relative  scarcities  between  the  principal  appropriation  categories 
affected  [Military  Pay,  Army  (MPA)  vs  Procurement  of  Equipment  and  Missiles, 
Army  (PEMA),  and  Operation  and  Maintenance,  Army  (CMA)]  or  the  many  aspects 
of  real  aircraft,  listed  in  Figure  l6,  which  were  not  modeled,  would 
have  to  be  considered  to  select  a  preferred  alteniative. 
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Fig.  16— Included  and  Excluded  Characteristics 
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